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I .  INTRODUCTION 

In  engineering,  economics  and  some  other  fields,  we  usually 
face  the  following  types  of  problems. 

(1)  With  the  information  obtained  from  an  observation  of  a 
phenomenon  in  the  past  and  present,  we  wish  to  predict  it  at  a 
future  time  with  the  best  accuracy  in  some  prescribed  sense. 

(2)  The  data  one  gets  in  observing  a  phenomenon  are  almost 
always  an  approximation  of  the  real  data,  which  is  due  to  the 
error  caused  in  the  process  of  observation;  for  Instance,  in  the 
measurement  of  a  quantity.  It  is  then  desired  to  estimate  the 
original  values  by  utilizing  the  observed  data  with  the  best  ac¬ 
curacy  in  a  certain  sense. 

(3)  With  a  given  input  time  series  and  a  linear  filter,  such 
as  an  electric  circuit,  a  unique  output  series  can  be  generated. 

In  case  the  filter  is  known,  and  only  the  output  series  is  observed, 
it  is  desired  to  recover  the  input  series,  and  to  determine  whether 
the  solution  to  this  inversion  problem  is  unique,  and  whether  the 
filter  is  physically  realizable,  that  is,  whether  the  input  series 
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at  any  time  can  be  expressed  solely  in  terms  of  the  observations 
of  the  output  series  made  in  the  past  and  present,  but  not  the 
future . 

To  treat  these  problems,  we  need  to  establish  some  mathemati¬ 
cal  and  statistical  models  of  time  series.  Then  we  can  describe 
these  problems  precisely,  and  provide  some  solutions. 

Let  the  triple  (n,E,P)  denote  a  probability  space,  where 
fl  is  a  sample  space,  £  is  the  a-algebra  of  all  events  under 
consideration,  and  P  is  a  probability  measure  on  2  .  Let  <C 
stand  for  complex  numbers.  For  any  complex  valued  random  variable 
X:fl-*C  ,  which  is  £ -measurable  by  definition,  the  expectation  of 
X  is  denoted  by  E(X)  ■  /xdP  .  The  Hilbert  space  H  ■  L^(C1,£,P) 

n 

of  all  the  (equivalence  classes  of)  random  variables  with  finite 
second  moments  can  be  constructed  as  usual  with  an  inner  product 
defined  by  (f,g)  •  E(fg)  .  The  "overbar"  denotes  complex  con¬ 
jugation.  Let  H  *  II 2  denote  the  corresponding  norm  of  H  .  For 

2 

convenience,  let  Lq (P)  stand  for  the  subset  of  functions  in 
2 

L  (fl,E,P)  with  means  zero. 

Let  X  •  {X(t),t€T}  denote  an  indexed  set  of  random  vari¬ 
ables.  If  T  is  the  set  Z  of  integers,  X  is  called  a  discrete 
parameter  time  series.  If  T  is  the  set  R  of  real  numbers,  X 
is  called  a  continuous  parameter  time  series.  If  every  X(t)  has 
a  finite  second  moment,  X  is  called  a  second  order  time  series. 

If  in  addition  X  is  of  means  zero,  i.e.,  E(X(t>)  ■  0  for  all 

t€T  ,  the  covariance  function  r:TxT-*C  is  defined  as  r(s,t)  ■ 
E(X(s)KTt7)  for  s,t€T  . 


In  order  to  study  the  analytical  properties  of  second  order 
time  series  with  means  zero,  we  first  classify  these  series  ac¬ 
cording  to  their  covariance  structure  as  follows.  A  list  of  more 
classes  of  time  series  can  be  found  in  [26]  and  [2]. 

1.  If  the  value  of  the  covariance  function  r  of  X  de¬ 
pends  only  on  the  difference  of  its  two  arguments,  that  is,  if 
r(s,t)  *  r(s+h,t+h)  for  all  s,t,h€T  ,  then  X  is  called  a  weakly 
stationary  time  series .  It  is  known  that  in  this  case  we  have  the 
representation 

r(s,t)  -  ;ei(s_t:>udM(u)  ,  (1) 

D 

where  D  «B  for  T  ■  B  ,  the  reals,  D  ■  [0,2")  for  T  “Z  , 
the  integers,  and  u  is  a  bounded,  monotone  increasing  nonnegative 
function  on  D  ,  called  the  spectral  function  of  X  .  It  is  known 
that  every  weakly  stationary  time  series  X  has  a  stochastic  in¬ 
tegral  representation 

X(t)  -  /eitXdZ(\)  ,  t€T  ,  (2) 

D 

2 

where  D  is  as  above,  Z:D~*Lq(P)  is  a  vector  valued  function 
with  finite  semi-variation  and  with  orthogonal  increments,  i.e., 
for  any  a^  <  <  a3  <  a^  , 


and  the  integral  is  in  the  sense  of  Dunford  and  Schwartz.  (See 
[10],  p.  323.) 

Note  that  without  the  restriction  that  the  time  series  X  is 
of  means  zero,  we  have  the  slightly  different  concepts  of  weak  and 
wide  sense  stationarity.  See  [2].  They  agree  under  our  hypotheses 


4 


2.  If  X  is  not  necessarily  weakly  stationary,  but  its  co- 
variance  function  r  admits  a  representation 

*(s,t)  ■  JJ*  e*su“icvdu(u,v)  ,  (3) 

DXD 


where  M  is  a  complex  valued,  positive  definite  function  of  two 
variables  with  finite  Vitali  variation  |nlv  on  DxD  defined  as 


|u|y  -  sup{^  S  iln(ti,tj)-M(ti_1,tj,)-ii(ti,tj_1)-Hx(ti_1,tj_1)| }  , 
then  X  is  called  a  strongly  harmonizable  time  series. [17] .  It 


was  proved  by  Loeve  [17]  that  every  strongly  harmonizable  time 
series  X  also  has  a  stochastic  integral  representation  (2), 
where  D  and  Z  are  as  before  except  that  Z  does  not  have 
orthogonal  increments,  as  in  the  weakly  stationary  case,  but 
satisfies  certain  other  conditions  generalizing  (2). 

It  is  easy  to  see  that  if  |i  concentrates  on  the  diagonal, 
that  is,  if  u(s,t)  -  0  for  s  +  t  ,  then  r(s,t)  depends  only 
on  the  difference  s-t  ,  so  that  X  is  weakly  stationary.  Thus 
the  class  of  strongly  harmonizable  time  series  is  an  extension  of 
the  class  of  weakly  stationary  time  series.  Again  m  Is  called 
the  spectral  function  of  X  . 

3.  If  the  function  ^  in  (3)  is  not  necessarily  of  Vitali 
variation  finite,  but  is  of  finite  Frechet  variation  defined  as 


then  X 


sup{  j 

i, 

|ail<l,|bj|<l,ai,bj€C}  , 

is  called  a  weakly  harmonizable  time  series  with 


S  aifi^[M(tl,tj)-M(ti.1,tp-H(tl,tj_1)-Hi(t 

j 


t' 

i-l»cj-l 

(5) 

a  as 


)] 
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its  spectral  function  [26].  By  the  definitions,  |u|pS|u|y  ,  and 
it  is  known  that  |n|p<«  does  not  imply  that  |^|v<*  ([7]),  so 

that  the  class  of  weakly  harmoni2able  time  series  is  an  extension 
of  the  class  of  strongly  harmonizable  time  series.  It  was  proved 

in  [26]  that  every  weakly  harmonizable  time  series  also  has  a 

2 

stochastic  integral  representation  (2),  where  2:D-*Lq(P)  is 
merely  a  vector  valued  function  which,  however,  always  has  a  finite 
semi- variation,  defined  in  ([10],  p.  321).  Note  that  the  right 
side  of  the  integral  representation  (2)  is  actually  the  Fourier 
transform  of  a  vector  valued  function  Z  .  The  minimum  require¬ 
ment  on  Z  for  the  integral  in  (2)  to  exist  in  the  sense  of  Dunford 
and  Schwartz  is  that  Z  must  be  of  finite  semi-variation.  Thus, 
by  relaxing  the  restrictions  on  Z  ,  we  extended  weakly  stationary 
series  to  strongly  harmonizable  and  then  to  weakly  harmonizable 
series,  and  no  more  extension  is  possible  if  the  representation 
(2)  is  to  hold. 

We  may  also  consider  multidimensional  time  series.  If  X(t)  • 
(X^(t) , . . . ,Xp(t))  ,  t€T  ,  is  a  p-dimensional  column  vector  of 

random  variables,  where  "  tr  "  indicates  the  transpose,  then  X 
is  called  a  p-dimensional  series.  If  for  any  p-vector  w  ■ 


(w^,...,Wp)  of  complex  numbers,  the  corresponding  time  series 

Y(t)  ■  ^w.X.(t),  t€T  ,  Is  weakly  stationary,  then  X  is  called 
jal  J  J 

a  p-dimensional  weakly  stationary  time  series.  Similarly,  we  have 


p-dimensional  strongly  and  weakly  harmonizable  time  series  defined 


in  the  same  way. 


The  concept  of  weakly  stationary  time  series  was  introduced 
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in  the  1930s.  Since  the  stationarity  assumption  is  reasonably 
close  to  the  reality  in  many  areas  in  practice,  and  since  powerful 
and  elegant  mathematical  tools  are  available  for  the  study  of 
weakly  stationary  time  series,  a  considerable  amount  of  important 
work  has  been  done  in  this  field.  In  the  case  of  strongly  harmoniz- 
able  time  series,  the  situation  becomes  more  complicated.  Not 
every  result  for  weakly  stationary  time  series  has  a  corresponding 
counterpart  for  strongly  harmonizable  series.  However,  the  stand¬ 
ard  analysis,  such  as  measure  theory  and  integration  theory,  is 
still  of  essential  importance  there.  It  was  known  that  given  any 
weakly  stationary  time  series  (X(t),t€T)  as  an  input  series,  and 
given  a  bounded  linear  projection  L  ,  which  is  clearly  a  (bounded) 
linear  operator,  it  can  be  proved  that  the  resulting  output  series 
Y (t)  ■  LX(t)  is  not  necessarily  strongly  harmonizable.  (See  [13], 
p.  183  and  [26],  p.  301.)  This  shows  that  the  classes  of  weakly 
stationary  as  well  as  strongly  harmonizable  time  series  are  not 
large  enough  for  operations  under  linear  transformations  such  as 
in  general  filtering  problems  (see  below).  This  is  indeed  a  short¬ 
coming  of  the  structure  of  these  two  classes. 

The  weakly  harmonizable  time  series  was  first  studied  under 
different  names  by  Bochner  [3]  and  Rozanov  [28]  in  the  1950s. 

More  work  in  this  area  can  be  found  in  [21],  [19]  and  [26].  The 
class  of  weakly  harmonizable  time  series  is  an  enlargement  of  the 
classes  mentioned  above,  and  it  is  closed  under  bounded  linear 
operators  [26].  This  closure  property  makes  this  class  a  more 
attractive  one  for  some  practical  applications  than  the  other  two. 
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However,  we  can  no  longer  apply  many  known  facts  in  standard  anal¬ 
ysis  as  in  the  previous  two  cases,  since  some  new  mathematical 
concepts  are  involved,  and  we  still  do  not  have  a  well-rounded 
theory  of  these  processes  for  an  analysis.  It  should  be  noticed 
that  some  results  can  be  extended  from  strongly  to  weakly  harmoniz- 
able  cases,  but  the  same  proofs  cannot  be  used,  and  a  different 
approach  is  often  necessary. 

II.  THE  LINEAR  FILTERING  PROBLEM 

We  now  recall  the  definition  of  a  linear  filter  for  our  work 
here.  A  linear  filter  L  is  a  mapping  L:X  *-»Y  or,  LX  *  Y  , 
where  X  *  [X(t):t€T]  and  Y  *  CY(t):t€T}  are  two  time  series 
with  T  *  Z  or  H  ,  such  that  (i)  For  any  a,b6R  and  time 
series  X  and  Y  ,  L(aX+bY)  •  aL(X)  +bL(Y)  ;  (ii)  For  any 
h€T  ,  with  a  time  series  X^  defined  by  Xh(t)  *  X(t4h),  t€T  , 
(LXh)(t)  -  (LX)(t4b)  ,  t<ET  . 

Condition  (i)  says  that  L  is  linear,  and  condition  (ii) 

says  that  L  commutes  with  translations  on  the  T  axis.  In  the 

2 

case  that  all  the  time  series  under  consideration  are  in  Lq(P)  , 
a  linear  filter  does  not  have  to  be  bounded,  i.e.,  IlLX (tjllj/'lx^t) II2 
need  not  be  bounded  for  t€T  .  For  instance,  one  can  have  dif¬ 
ferential  filters  which  need  not  be  bounded.  A  study  of  the 
general  concept  of  a  linear  filter  on  a  second  order  homogeneous 
time  series  on  a  globally  symmetric  index  set  can  be  found  in 
Yaglom  [29]  and  Hannan  [12],  which  extend*  the  work  of  Masanl  [18] 
where  the  index  set  is  the  real  line.  However,  they  will  not  be 


■fjffilir  ■  *i 


considered  here. 

Next  we  discuss  Che  problem  mentioned  ac  the  beginning  of 
Section  I.  Let  X  *  {X(n),n€Z}  be  a  time  series,  and  let  L  be 
a  polynomial  filter,  also  called  a  moving  average  filter,  as  fol¬ 
lows  : 

LX(n)  -  Sa.X(n-j)  ,  n€Z  ,  (6) 

j-0  J 

where  aQ,...,a^  are  constants.  If  Y  *  [Y(n),n€Z]  is  such 
that  Y(n)  -  LX(n)  for  all  n€Z  ,  Y  is  called  an  output  series, 
and  X  is  called  an  input  series.  The  inversion  problem  is  to 
solve  the  equation  Y  -  LX  for  the  series  X  with  given  Y  and 
L  .  Let  f  be  a  function  defined  by 

f(t)  -  Sa.e1^  ,  0*C<2tt  .  (7) 

j-0  J 

f  is  called  the  spectral  characteristic  of  the  polynomial  filter 
L  defined  by  (6),  and  it  is  also  called  the  "frequency  response" 
in  the  engineering  literature. 

The  following  result  is  from  [20] .  Let  Y  be  a  given  weakly 
stationary  time  series  with  a  spectral  function  m  ,  and  let  L 
be  a  polynomial  filter  defined  by  (6).  Let  Q  denote  the  set  of 
zeros  of  f  in  the  interval  W  -  [0,2tt)  ,  i.e.,  Q  -  {t€W:f(t)«0} 
Then  there  exists  a  weakly  stationary  time  series  X  -  Cx(n),n€Z} 
such  that  Y(n)  -  LX(n)  for  all  n€Z  if  and  only  if  the  follow¬ 
ing  two  conditions  are  satisfied: 

(i)  J*dn<t)  -  0  , 

Q 


(8) 
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(ii)  „ 


I  7-JT7Z^(t)  <  -  . 

w-q  if(t>r 


(9) 


If  both  conditions  are  satisfied,  then  the  solution  series  X  to 
the  inversion  problem  is  unique  if  and  only  if  Q  is  empty.  If 
Q  is  not  empty,  there  exists  only  one  solution  belonging  to  the 
closed  span  of  all  square  integrable  functions  relative  to  u 
under  the  norm  and  whose  spectral  function  v  satisfies  the 

condition  jdv(t)  =  0  .  Furthermore,  if  all  the  roots  of  the 


characteristic  polynomial  P(t) 


£a,tJ 


j-0 


J' 


are  outside  the  unit 


circle  (t€C:|t|  **1]  ,  then  the  filter  L  is  physically  realizable. 

00 

Actually,  with  1/P(t)  *  2  b  tn  being  the  Taylor  series  expan- 


n-0 

sion,  we  have  the  expression  X 


2  b  Y 


n  m-n  * 


for  all  m€Z  . 


m  n-0 

However,  if  some  roots  are  inside  and  none  is  on  the  unit  circle, 

then  L  is  not  physically  realizable.  In  this  case,  the  function 

00 

1/P(t)  has  a  Laurent  series  expansion  2  bntn  >  and  the  ^uture 


n=-° 


m€Z  . 


values  of  Y  are  clearly  involved  in  the  expression 

m 

X  -  2  b  Y 

m  _  n  m-n  * 

n--» 

In  case  both  X  and  Y  are  of  continuous  parameter,  results  as 
described  below  are  the  analogs  of  the  above. 

One  defines  an  integral  filter  L  on  a  continuous  parameter 
time  series  X  ■  Cx(t),t€R}  as  follows: 


LX(t)  -  ,fg<u)X(t-u)du  , 


t€*  ,  (10) 


where  g  is  a  Lebesgue  integrable  weight  function  over  R  ,  and 
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the  integral  on  the  right  side  of  (10)  with  a  vector  integrand  is 
in  the  sense  of  Bochner  [9].  The  spectral  characteristic  F  as¬ 
sociated  with  L  is  now  taken  as  the  Fourier  transform  of  g  , 
i.e. , 

F(t)  =  JVltug(u)du  ,  t€R  .  (11) 

R 

With  L  and  F  just  defined  and  with  W  =  R  ,  all  the  results 
listed  above  for  polynomial  filters  still  hold,  except  the  last 
part  concerning  the  physical  realizability  of  L  .  For  the  un¬ 
bounded  linear  filter  defined  by  a  difference-differential  operator 
and  also  for  some  more  general  filters,  similar  results  have  been 
obtained  by  the  author  in  [5J.  In  the  case  when  both  input  and 
output  series  X  and  Y  are  strongly  harmonizable ,  the  corre¬ 
sponding  work  was  done  by  Kelsh  [15]. 

Since  the  spectral  function  u  of  the  output  series  Y  is 
now  defined  on  Rxl  ,  the  necessary  and  sufficient  conditions 
(8)  and  (9)  for  the  existence  of  a  solution  X  ,  which  is  also 
strongly  harmonizable,  to  the  equation  Y  =  LX  with  L  defined 
by  (6)  should  be  replaced  by 

(i)'  Mt(Q*Q)  “  0  ,  (12) 

(ll)'  |f(u)f(:v)|d'tfle<u’v>  <  *  ’  <13> 

QcxQc 

where  |ult(u,v)  is  the  total  variation  of  n  on  the  rectangle 

(-•,u)  x  (-«,v)  ,  and  |nlc(Q*Q)  is  that  on  the  set  Q*Q  .  The 

uniqueness  part  is  the  same  as  in  the  weakly  stationary  case, 

except  that  the  condition  J*dv(t)  ■  0  should  be  replaced  by 

Q 
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lvlt(QxQ)  “  0  ,  where  v  is  the  spectral  function  of  X  .  Finally, 
the  physical  realizability  part  is  identical  with  that  in  the 
previous  case.  A  similar  result  holds  for  the  integral  filters 
defined  by  (10). 

It  is  easy  to  see  that  if  ia  concentrates  on  the  diagonal, 
then  (i)'  and  (ii)  '  reduce  to  (i)  and  (ii).  In  the  multidimensional 
case,  the  work  becomes  more  involved.  For  the  multidimensional 
polynomial  filters,  Kelsh  [15]  solved  the  problem  for  a  sub-class 
of  multidimensional  strongly  harmonizable  time  series,  which  was 
called  the  factorizable  spectral  measure  series,  and  which  con¬ 
tains  all  the  multidimensional  weakly  stationary  series  as  a  proper 
subset.  Since  the  frequency  characteristic  matrix  may  be  singular, 
the  generalized  inverse  of  a  matrix  introduced  by  Moore  and  Penrose 
[23]  played  an  important  role  there. 

In  the  case  when  Y  is  weakly  harmonizable,  the  spectral 
function  n  of  Y  is  only  of  Frechet  variation  finite.  The  above 
condition  (ii) 7  can  no  longer  be  used,  since  |n(t(u,v)  will  gen¬ 
erally  be  infinite.  Instead,  we  use  the  following  definition  of 
a  reproducing  kernel  Hilbert  space  associated  with  the  spectral 
function  (cf.  [1]). 

Let  f  (• ,  •  )  :lRxll  -•  C  be  a  positive  definite  function.  Let 
H  be  the  set  of  linear  combinations  of  the  functions  f(a,*):®**  C  , 

i.e.,  H  ■  sp{  f  (a,  •  )  :a€]R}  .  For  any  two  elements  g  *  Sb,f(a,  ,•) 

j-1  j  J 

h  *  ^cL.f(a/,*)  in  H  ,  define  an  inner  product  by 
k-1  *  K 
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<g,h)  -  £  E  b.c,f  (a.  ,a/)  . 
j=l  k=l  3  3 

Let  H*  be  the  closure  of  H  under  (*,•)  .  Then  H*  is  called 
a  reproducing  kernel  Hilbert  space  associated  with  the  function 
f  ,  and  f  is  called  its  reproducing  kernel.  Now  we  regard  f 
as  a  vector  valued  function,  f  :a  H*  ,  for  all  a€H  .  A  theory 
of  integration  of  a  scalar  function  with  respect  to  a  vector  in- 

1  u* 

tegator  is  available,  see  ([10],  p.  323).  Let  L  (f  )  be  the 
set  of  all  functions  g:E-*  (t  which  are  integrable  relative  to 
f:]R  -*H*  in  this  sense. 

Now  let  Y  be  a  weakly  harmonizable  time  series  with  spectral 

function  u  .  Let  L  be  a  linear  filter,  either  a  polynomial  or 

an  integral  filter,  with  the  spectral  characteristic  F  .  The 

necessary  and  sufficient  conditions  for  the  existence  of  a  weakly 

harmonizable  time  series  X  satisfying  LX  *  Y  are  as  follows: 

(i)"  HF(Q*Q)  -  0  , 

(ii)"  X  /F6L1 (HH*)  . 

Qc 

For  the  uniqueness  and  the  physical  realizability,  we  have  the 
same  situation  as  before.  Similar  results  hold  for  certain  multi¬ 
dimensional  filters  (cf.  [5]).  It  should  be  noted  that  the  above 
conditions  and  hence  the  result  for  weakly  harmonizable  time  series 
is  not  the  same  as  that  for  the  strongly  harmonizable  time  series. 
However,  both  agree  when  specialized  to  the  stationary  case. 
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III.  OPTIMAL  SIGNAL  ESTIMATION 

Next  we  consider  the  problem  of  filtering  signal  from  noise. 

Let  S  *  CS(t):t€]R}  and  N  ■  [N(t):t€]R}  be  the  signal  and  noise 
time  series  respectively,  and  let  X  •  £x(t):t€]R}  be  the  out¬ 
put  series  which  consists  of  both  signal  and  noise,  i.e.,  X  ■  S+N  . 
We  only  consider  the  continuous  parameter  case  here.  If  the  signal 
S  and  the  noise  N  are  not  observable,  but  the  output  X  is 
observable,  it  is  desired  to  estimate  S  by  a  best  linear  filter 
L  operating  on  X  in  the  mean  square  error  sense,  i.e.,  with 
the  error  !|S(t)  -  LX (t ) i] 2  minimized  for  t€]R  .  We  assume  that 
the  series  S  and  N  are  of  the  same  type,  and  some  conditions 
on  their  spectral  functions  are  needed  for  obtaining  correspond¬ 
ing  results . 

In  the  case  that  the  noise  series  N  is  Gaussian,  the  work 
in  this  problem  can  be  found  in  [14]  and  [22].  In  the  weakly 
stationary  case,  the  problem  was  treated  by  Grenander  [11],  as 
follows.  Let  S  ,  N  and  X  be  weakly  stationary.  Then  X  will 

have  the  integral  representation  X(t)  ■  j'eit^dZ(X)  ,  where 

R 

2 

Z:R-*  Lq(P)  is  a  vector  valued  function  with  orthogonal  incre¬ 
ments.  Let  vAg  and  ^  be  the  spectral  functions  of  series  S 
and  N  .  If  ^  and  nN  are  absolutely  continuous,  i.e.,  if 
the  derivatives  fg(t)  ■  dHg(t)/dt  and  fN(t)  ■  dpN(t)/dt  exist, 
and  if  S  and  N  are  uncorrelated,  i.e., 

rs>N(u»v)  “  E(S(u)N(v))  -  0  ,  u,v€lR  , 

then  the  best  linear  filter  S  in  the  mean  square  error  sense, 
which  is  also  called  an  optimal  filter,  is  given  by  the  formula 


14 


fs(M  ltX 

S(t)  dZ(X)  ’  t63R  * 

If  the  integrand,  which  is  called  a  response  function  or  a  filter 

function,  can  be  approximated  by  an  expansion 
fs(X)  M  iakX 

>«„(*>  '  #2  i^k* 
then  we  have 

m 

S(t)  -  S  c,  X(t+a.  )  . 
k-1  *  K 

In  general,  if  the  signal  and  the  noise  are  not  necessarily 
uncorrelated,  but  are  weakly  stationarily  correlated,  i.e., 

rS,N^u,v^  "  fS,N^u“V^  “  ^  ^  *  u,v€R  , 

and  if  the  function  us  N:R  -C  is  absolutely  continuous,  so  that 
one  has 

?s,N(u-v)  ->1(u'v)Xfs>N(Mdx  , 

the  solution  to  the  filter  problem  is  given  by  the  expression 
r  fS(X)+*S  N(X)  itX 

S(t)  "  l  fg ft (T )WeTI”^nT  C  dZ(X)  *  t€3R  ‘  (14) 

Note  that  in  the  above  results,  one  has  to  assume  that  all  the 
spectral  functions  and  the  cross-spectral  function  are  absolutely 
continuous.  If  this  is  not  the  case,  the  results  become  more 
complicated. 

When  the  series  S  ,  N  and  X  are  of  Cramer  class  as  de¬ 
fined  in  [8],  which  contains  the  class  of  all  strongly  harmoniza- 
ble  time  series,  and  when  S  and  N  are  uncorrelated,  then 
similar  results  were  obtained  in  [27].  Without  assuming  that  S 
and  N  are  uncorrelated,  Kelsh  [15]  considered  the  same  problem 


for  multidimensional  Cramer  class  series,  and  got  the  corresponding 
result  using  the  technique  essentially  due  to  Rao  [27].  For  one 
dimensional  strongly  harmonizable  series  S  ,  N  and  X  ,  Kelsh's 
result  can  be  stated  as  follows. 

Let  ~*(C  be  the  spectral  functions  of 


S  and  N 


Mg  ^:KxR-*c  be  the  cross -spectral  function,  and  let 
9  * 


€  be  defined  by  Mg  N(u,v)  "  N^v*u^  »  for  u»veiR 
optimal  filter  is 


Then  the 


S(t) 


jF(X)dZ  (X) 

R 


t€R 


(15) 


where  F:H**<E  is  a  solution  to  the  set  of  integral  equations 

Kx-f(U)e  d^s'HiN‘HiS,N‘HiS,H)(u,v)  “  j^itU  iSVd(^s'tvlS,N)(u’v)  » 
for  all  se» 


In  general,  it  is  not  easy  to  solve  this  system 
of  integral  equations  analytically.  However,  if  the  spectral 
functions  ug  ,  mn  and  Mg  N  are  absolutely  continuous,  expres- 
sion  (15)  can  be  reduced  to  an  explicit  form  as  in  (14). 


IV.  SAMPLING  A  HARMONIZABLE  PROCESS 

Next  we  discuss  the  sampling  problem  of  the  continuous  pa¬ 
rameter  time  series.  When  we  study  a  time  series  in  practice,  it 
is  sometimes  physically  difficult  or  economically  undesirable  to 
observe  the  whole  series.  It  is  then  required  to  sample  it  at 
only  finitely  many  times,  and  to  estimate  the  original  series  from 
the  observed  samples.  Sampling  theorems  are  very  important  in 
many  fields  in  practice,  such  as  the  coemunlcation  and  information 
theory. 
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The  following  result  Is  called  Kotel 'nikov -Shannon  formula, 
and  Is  an  abstraction  of  a  classical  (nonstochastic)  result  due 
to  Cauchy  [4] . 

If  X  ■  Cx(t):t€]R}  is  a  weakly  stationary  time  series  with 
spectral  function  u  which  is  supported  by  a  bounded  interval 
(-l/2h,l/2h)  ,  h  >0  ,  i.e.,  it  is  constant  in  (-“,-l/2h]  and 
[l/2h,~)  ,  then 

x(c>  ■  JS  J./(nh)''n^(-VhtA/hl .  .  <16> 

where  the  convergence  on  the  right  side  of  (16)  is  in  the  sense  of 
mean  square.  This  formula  gives  a  periodic  sampling  theorem,  where 
one  observes  the  time  series  at  the  periodic  points  t  -  nh  , 
•N<n*N  ,  and  the  random  variable  X(t)  at  any  time  t  can  be 
estimated  by  (16)  by  taking  large  enough  N  . 

If  X  is  weakly  stationary  but  its  spectral  function  M  is 
not  necessarily  supported  by  a  compact  set,  then  the  following 
results  of  Lloyd  [16]  hold. 

If  n  has  an  open  support  S  such  that  the  sets  [S+nhJ  , 
n€Z  ,  are  mutually  disjoint,  i.e.,  no  two  values  in  S  differ  by 
an  integer  multiple  of  1/h  ,  then 

X(t)  -  l.i.m.  £(1  -  jn|/N)X(nh)K(t-nh)  ,  t€H  ,  (17) 

N—  n— N 

where  K(t)  ■  hJ*e^TT^tdX,  t€R  ,  and  l.i.m.  denotes  the  con- 
S 

vergence  in  mean  square.  If  S  is  a  finite  union  of  intervals, 
or  if  sup  | tK(t) |  <  •  ,  then 
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N 

X(t)  -  l.i.m.  S  X(nh)K(t-nh)  ,  t€R  .  (18) 

N—  n— N 

The  problem  of  sampling  for  strongly  harmonlzable  time  series 

has  been  studied  by  Rao,  Plranashvlll  and  Pourahmadl  (cf.  [27],  [24], 

[25]).  The  formula  (16)  was  obtained  by  Plranashvlll  [24]  for 

strongly  harmonlzable  time  series  whose  spectral  function  has  a 

bounded  support  in  R  xR  .  The  formulas  (17)  and  (18)  have  been 

extended  to  strongly  harmonlzable  time  series  by  Rao  [27]  and 

Pourahmadl  [25].  Let  X  be  a  strongly  harmonlzable  time  series 

with  spectral  function  n  ,  and  let  v  be  a  function  defined  by 

v(s)  »  |u  L((-",s),R)  ,  s€]R  .  If  v  has  an  open  support  S  such 

that  the  sets  [S+n/h],  n€Z  ,  are  mutually  disjoint,  then  (17) 

holds.  If  S  is  a  finite  union  of  intervals,  or  if  sup  |tK(t)| 

-"B<t<* 

<  -  ,  then  (18)  holds. 

In  the  case  where  X  ■  Cx(t):t€]R}  is  weakly  harmonlzable 

with  spectral  function  F  ,  we  have  the  following  result  due  to 

Chang  and  Rao  [6].  Given  any  e  >0  ,  if  there  exists  a  bounded 

Borel  set  A  *  (Ae )  c  p.  such  that 

J\f  dF(u,v)  <  e/4  ,  V  B  c  Ac  (Ba  Borel  set), 

B*B 

and  if  Oq  »  diameter  of  A  ,  then  for  any  h<TT/cjQ  ,  one  has  an 
N  (  »N€  t  )  such  that 

ilx(t)  -  n j^X (nh )  J  If 2  <  C(t)[0T-hoo)N] -1  +e  , 

where  0<C(t)<*  is  bounded  for  t  in  bounded  sets.  If  the 

spectral  function  F  has  a  bounded  support,  then  we  can  set  «  ■  0 

The  above  formula  actually  gives  an  estimation  of  the  error  In  the 
mean  square  sense. 


V.  A  NUMERICAL  ILLUSTRATION 

In  Chis  section  we  give  a  numerical  example  which  deals  with 
the  data  of  a  nonstationary  time  series.  The  data  is  from  a  tape 
which  contains  digitized  acoustic  data  of  the  time  series  output 
for  one  beam  of  a  multibeam  sonar.  The  time  series  is  primarily 
dominated  by  energy  emitted  by  a  transiting  merchant  ship,  and  it 
contains  significant  energy  over  a  board  range  of  frequencies.  The 
tape  is  very  kindly  provided  by  Dr.  D.  F.  Gingras  of  the  Naval 
Ocean  System  Center  at  San  Diego,  California.  Since  the  data  is 
used  solely  for  the  purpose  of  illustration,  no  specific  details 
will  be  discussed. 

A  set  of  4000  data  is  read  in  from  the  tape  and  stored  in  a 
vector  Y  of  real  numbers.  Let  *o,,,,,a6  be  the  real  coeffi¬ 
cients  to  be  determined.  For  each  8*n‘4000  ,  we  define  the 
"error" 

cn  "  Y<n)  "  ^  ««Y(n-J)  . 
n  j-0  J 

4000  2 

Minimizing  the  sum  I  ef  relative  to  the  a.'s  ,  we  obtain  the 

n*8  n  1 

values  for  the  filter  coefficients  *q>***»*6  .  Correct  to  three 
decimal  places,  these  are  as  follows: 

aQ  -  0.852  a3  -  -0.087 

ax  -  -0.463  a4  -  0.007 

*2  "  0*227  a5  -  0.002 

a6  -  -0.100  . 

With  the  given  sequence  Y  as  the  output  series,  and  with  the  co¬ 
efficients  Sj  obtained  as  above,  we  can  consider  the  filtering 
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problem  Y(n)  -  LX(n)  ■  £a,X(n-j)  ,  for  n*7  ,  where  X  is  an 

j-0  J 

unknown  input  series.  Note  that  if  we  assume  that  Y(n)  -  0  for 
all  n*0  ,  the  problem  becomes  quite  simple.  This  is  not  assumed 
here. 

The  characteristic  polynomial  of  the  filter  L  is  of  the 

form  P(t)  -  ^  a.t-^  .  The  roots  t,,...,t..  of  P  can  also  be 
j-0  J  10 

computed.  These  are  as  follows: 


tL  -  1.295  t3,t4  -  0.501  ±  1.357i 

t2  -  -1.746  t5,t6  -  0.739  ±1.1181  . 

Since  all  these  roots  lie  outside  the  unit  circle,  the  filter  L 
is  physically  realizable.  To  compute  the  values  for  the  sequence 
X  ,  we  need  to  expand  the  rational  function  1/P  using  the  Taylor 
series  method.  With  the  coefficients  b^,b2,...  thus  determined, 

we  can  use  the  formula  X(m)  -  S  b  Y(m-n)  to  obtain  the  input 

n-0  n 

series  X  .  The  first  24  b's  ,  correct  to  three  decimal  places, 
are  as  follows : 


b0 

-  1.177 

bfi  -  0.136 

b12 

-  0.018 

b18 

-  0.003 

bl 

-  0.640 

b?  -  0.147 

b13 

-  0.025 

b19 

-  0.004 

b2 

-  0.033 

bg  -  0.049 

b14 

-  0.015 

b20 

-  0.003 

b3 

-  -0.032 

b9  -  -0.003 

b15 

-  0.003 

b21 

-  0.001 

b* 

-  0.030 

b1Q  -  0.003 

b16 

-  0.000 

b22 

-  0.000 

bj  -  0.019  bj^  -  0.008  b17  - 


0.002 


0.000 
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A  set  of  two  hundred  values  (from  the  same  data  records)  of  X 
and  Y  ,  correct  to  two  decimal  places,  is  given  in  Table  1,  and 
the  graphs  for  both  series  X  and  Y  with  these  values  are  plot 
ted  in  Figure  1  for  comparison. 


( 


I 


X  : 
Y  : 

5o.6«  46.50 

3b. 00  26.00 

17.34 

6.00 

y.2b 

31.no 

51.16 

27.00 

62.02 

33-00 

o3-?4 

28.00 

44.35 

18.00 

X  : 

22. SI  -24.86 

-24.45 

-5.87 

-0.74 

-0.53 

-25.60 

-62.52 

Y  : 

4. no  -20.00 

-13. *0 

-7.no 

-15.00 

-7.no 

-22.00 

-40.00 

X  : 

-2o.0q  -12. 47 

-20. 3* 

-26.02 

-42. ^4 

-53. 16 

-3“.H2 

-14.00 

Y  : 

tt.OO  -in.no 

-18.0* 

-10. *0 

-z'7 . 00 

-23.00 

-in.no 

-8.00 

X  : 

-21. S2  -10.60 

11.  OH 

31. HI 

28.22 

15.36 

1.51 

16.35 

Y  : 

-in.no  -6.00 

20.00 

24.no 

17.no 

8.00 

0.00 

14.00 

X  : 

SI. 61  32.«o 

20.11 

48.21 

6. 16 

-25.86 

7.62 

I0.30 

Y  : 

33-00  5.00 

I’.OO 

20.n0 

-13.00 

-18.no 

11.00 

1.00 

X  : 

-14.44  -36.44 

-21.63 

-22.06 

-14. 53 

12.68 

3.02 

-43.41 

Y  : 

-’0.00  -26.00 

-7.00 

-13. *0 

-4.00 

15. *0 

-4.00 

-31.00 

X  : 

-37.65  -3S.71 

-27.08 

-28.84 

-0.38 

41  .ns 

42.43 

-26.40 

Y  : 

-10.00  -21.00 

-10.00 

-18.00 

4. no 

35.nO 

23.00 

-32.00 

X  : 

-54.60  -24.72 

-25.02 

-32.84 

-2.06 

1.84 

-20.87 

-56.66 

Y  : 

-24.00  -j.OO 

-20.00 

-21.00 

5.00 

n.no 

-11.00 

-36.00 

X  : 

-20.33  46.23 

63-53 

51.70 

63-24 

57.73 

21.57 

42.62 

Y  : 

-1.00  45.00 

31.00 

211.OO 

42.00 

32.00 

5.00 

30.no 

X  : 

45.50  16.25 

8. 15 

-42.00 

-75.47 

-60. 6'7 

-43.61 

-63.o5 

Y  : 

13-00  -4.00 

0.00 

-46.00 

-46.no 

-31. *0 

-27.00 

-43.00 

X  : 

-o8.65  -62.00 

-52.70 

-25.50 

-5.8b 

-in. 84 

5.56 

18.36 

Y  : 

-35.00  -20.00 

-10.00 

0.00 

4.00 

-2.00 

I7.no 

17.no 

X  : 

-16.33  -54. q6 

-13-7" 

20.08 

42. 14 

44.10 

43.42 

24.88 

Y  : 

-15.00  -33. n0 

0.00 

22.00 

23-*0 

28.00 

23. *0 

14.0° 

X  : 

34.20  24.15 

2.77 

-0.71 

30.88 

45.18 

15.14 

1.18 

Y  : 

25.00  4.00 

-7.00 

-4.00 

21.no 

22.00 

-tt.OO 

-1.00 
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TABLE  1  (Continued) 


X 

; 

-8.23 

-SO. 02 

-86.61 

-66.5'' 

-0.76 

10.18  0.82  -33-10 

y 

: 

-8.00 

-67.00 

-S  ? . 00 

-32.00 

13. 0" 

0.00  5.00  -23.00 

X 

; 

-26.27 

20.70 

32.36 

20. SO 

21. 57 

-10.05  -56.82  -70.72 

y 

• 

2.00 

22.00 

15.  n0 

IS. 00 

9.80 

-12.00  -38.00  -60.00 

X 

• 

-26.3? 

i".3a 

17.36 

16.21 

01.62 

51.27  11.86  50.26 

y 

• 

-3.00 

0.80 

a. 00 

0.00 

56.00 

26.00  1.80  26.00 

X 

i 

so.au 

32o6 

10. 57 

36.31 

58.60 

22.01  -30.16  -50.60 

y 

• 

26.00 

0.00 

-3.00 

26.^0 

32.00 

-3.00  -31.80  -32.0" 

X 

« 

-io.69 

-31.0a 

-us. 00 

-SO. 65 

.60.66 

-21.23  -5.85  -7.21 

y 

• 

8.08 

-31.00 

-30.00 

-30.00 

-16.00 

-2.00  3. 08  -3.00 

X 

-17. OS 

-to. 01 

12.31 

38. 61 

61.67 

62.31  3?.32  20.33 

y 

-6.00 

-3.*0 

10.00 

27.00 

22.00 

25.00  20.00  8.00 

X 

• 

• 

-«.H9 

26.73 

77.68 

80.61 

21. 7? 

-25.62  -66. O7  -57 • 56 

Y 

: 

-’3.00 

23.00 

67.00 

3S.00 

-7.00 

-22.00  -28.00  -37.00 

X 

» 

• 

-66 .  98 

-S8.60 

-65.03 

-22.8? 

12.66 

-6.08  -33- O7  -12. 72 

y 

• 

• 

-66.08 

-36.00 

-26 . On 

-6.00 

20.00 

-7.80  -15.00  8.00 

X 

• 

US. 63 

8S.60 

OS.  00 

66.63 

20.03 

-6Q.77  _ud. 37  21.58 

y 

■ 

62.00 

S6.00 

S2.00 

28.00 

12.80 

-67.00  -21.00  19.00 

X 

• 

60.26 

*3.30 

26.50 

S.22 

-28.68 

-52.19-31.25  -6.8? 

y 

• 

• 

33-00 

7."o 

13.00 

1.08 

-10.00 

-36.00  -16.00  -5.00 

X 

j 

20. 2S 

13- 12 

-6.66 

6.62 

35.86 

61.95  -8.61  -62.37 

y 

• 

15.  On 

2.00 

-6.00 

15.00 

28.00 

22.00  -21.08  -27.00 

X 

j 

-SU.06 

-22.13 

16.36 

1.63 

-36.77 

-57.53  -57.1,  -26.63 

y 

: 

-31.00 

-3-00 

12.00 

-11. "0 

-26.00 

-29.8Q  -25.00  -6.00 

100 
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